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DESCRIPTION 
ARANGE IMAGE SENSOR 

TECHNICALFIELD 

The invention relates a range image sensor for capturing an image of 
an object space to generate a range image in which each image element 
includes, when at least a physical object is in the object space, a 
corresponding distance value up to the physical object. 

BACKGROUND ART 

A displacement measuring device described Japanese Laid- op en 
Patent Publication No. H09-257418 scans a light beam irradiated to an 
object to be measured and then measures a distance up to a displacement 
surface of the object with respect to a reference surface of the object based on 
triangulation. 

However, the device is not suitable for such purposes as, for example, 
tracing the locus of motion of a physical object in an object space. Because it 
is necessary to scan the whole object space with the beam in order to trace 
the locus and comparative long time is taken until the device finishes 
scanning the whole object space with the beam. 

This problem can be solved by a range image sensor according to our 
another invention (cf. Japanese Patent Publication No. 2004-272001). The 
sensor comprises a light source which emits intensity modulated light 
toward an object space and a light detecting element with a photosensitive 
array disposed to face the object space. While the intensity modulated light 
is emitted toward the object space, a range image is constructed based on the 



WO 2006/030989 



2 



PCT/JP2005/017569 



output of the light detecting element. When at least a physical object is in 
the object space, each image element in the range image has a corresponding 
distance value up to the physical object. Each distance value is obtained 
from a difference between the phase of the light emitted from the light source 

5 and the phase of the light received by a corresponding pixel of the light 
detecting element. A period of time (hereinafter referred to as an 
"integration period") during which each pixel mainly integrates an electric 
charge in response to the light is set to a shorter time than one period of the 
intensity modulated light. 

10 If the integration period is comparative long time, it is possible to 

reduce shot noise occurred at the photosensitive array to raise the S/N ratio. 
But, the light detecting element becomes easy to be saturated under 
circumstances which includes a lot of ambient light noise (e.g., solar 
radiation). On the other hand, if the integration period is comparative 

15 short time, the light detecting element becomes hard to be saturated. 
However, the S/N ratio falls and it is difficult to obtain correct distance. 

DISCLOSURE OF THE INVENTION 

It is therefore an object of the present invention to raise accuracy of 
20 distance measurement by raising the S/N ratio while preventing saturation 
of the light detecting element. 

A range image sensor of the present invention comprises a light 
source, a light detecting element, a sensor control stage and an image 
construction stage. The light source emits intensity modulated light toward 
25 an object space according to a modulation signal of a specific frequency. The 
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light detecting element has photosensitive units disposed to face the object 
space. Each of the photosensitive units receives light from the object space 
during an integration period shorter than one period of the specific frequency, 
and generates an electric charge in response to a quantity of light from the 

5 space. The sensor control stage controls to synchronize each integration 
period of the photosensitive units with a specific phase of the modulation 
signal. After a detection period corresponding to one or more periods of the 
specific frequency, the sensor control stage also controls to pick up each 
electric charge generated and integrated in the light detecting element from 

10 the light detecting element. The image construction stage calculates a 
distance value for each image element in a range image based on each 
electric charge picked up by the sensor control stage, and then constructs the 
range image. When at least a physical object is in the object space, the 
distance value represents distance up to the physical object. The detection 

15 period includes different detection periods. The image construction stage 
calculates a distance value for each image element in the range image based 
on each electric charge picked up after a specific detection period of the 
different detection periods by the sensor control stage. The specific 
detection period is one of one or more detection periods during which the 

20 light detecting element does not reach saturation, and is one detection period 
during which a value related to the quantity of light received from the object 
space becomes maximum of that of the one or more detection periods. 

In this configuration, since a distance value for each image element 
in the range image is calculated based on each electric charge picked up in 

25 synchronization with the specific detection period and then the range image 
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is constructed, it is possible to raise the S/N ratio while preventing 
saturation of the light detecting element and raise accuracy of distance 
measurement. 

In a preferred embodiment, the sensor control stage controls to 

5 synchronize each integration period of the photosensitive units with each of a 
set of phases different from each other in the modulation signal. The sensor 
control stage also controls to pick up one set of electric charges corresponding 
to the one set of phases at every image element in the range image after at 
least the specific detection period of the different detection periods. The 

10 image construction stage calculates a distance value for each image element 
in the range image based on one set of electric charges picked up at every 
image element in the range image after the specific detection period. In 
this configuration, a distance value can be calculated based on one set of 
electric charges corresponding to the one set of phases. 

15 In an enhanced embodiment, the sensor control stage controls to pick 

up one set of electric charges corresponding to the one set of phases at every 
image element in the range image after each of the different detection 
periods. The image construction stage selects the specific detection period 
out of the different detection periods. The image construction stage also 

20 calculates a distance value for each image element in the range image based 
on one set of electric charges picked up at every image element in the range 
image after the specific detection period. The specific detection period is 
one of one or more detection periods during which a value obtained from one 
set of electric charges picked up at every image element in the range image 

25 after each of the different detection periods does not exceed a value 
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predetermined based on saturation level of the light detecting element, and 
is one detection period during which the value obtained from one set of 
electric charges becomes maximum of that of the one or more detection 
periods. According to this invention, it is possible to raise the S/N ratio 
while preventing the saturation of the light detecting element and raise the 
accuracy of distance measurement. 

In an alternate embodiment, the specific detection period is one of 
one or more detection periods during which a value of one set of electric 
charges picked up at every image element in the range image after each of 
the different detection periods does not exceed a prescribed value 
corresponding to the saturation level, and is one detection period during 
which the value of one set of electric charges becomes maximum of that of 
the one or more detection periods. According to this invention, it is possible 
to raise the S/N ratio while preventing the saturation of the light detecting 
element and raise the accuracy of distance measurement. 

In another alternate embodiment, the specific detection period is one 
of one or more detection periods during which a value of maximum electric 
charge for each of one set of electric charges picked up at every image 
element in the range image after each of the different detection periods does 
not exceed a maximum threshold value predetermined based on the 
saturation level, and is one detection period during which the value of 
maximum electric charge becomes maximum of that of the one or more 
detection periods. According to this invention, it is possible to raise the S/N 
ratio while preventing the saturation of the light detecting element and raise 
the accuracy of distance measurement. 
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In other alternate embodiment, the specific detection period is one of 
one or more detection periods during which a mean value with respect to one 
set of electric charges picked up at every image element in the range image 
after each of the different detection periods does not exceed an average 

5 reference value predetermined based on the saturation level, and is one 
detection period during which the mean value becomes maximum of that of 
the one or more detection periods. In this configuration, it is possible to 
consider intensity modulated light from the object space and ambient light 
component to distinguish whether or not the light detecting element is 

10 saturated. 

In another enhanced embodiment, when the specific detection period 
is not selected or a first specific detection period as the specific detection 
period is shorter than prescribed length, the image construction stage selects 
a second specific detection period out of the different detection periods at 

15 every one or more phases of the one set of phases with respect to a specific 
image element for which the specific detection period is not selected or the 
first specific detection period shorter than the prescribed length is selected. 
The image construction stage then corrects one or more values obtained from 
one set of electric charges picked up with respect to the specific image 

20 element after the second specific detection period based on a rate of length of 
the corresponding second specific detection period. The image construction 
stage then calculates one set of values and calculates a distance value for the 
specific image element based on the one set of values. The second specific 
detection period is one of one or more detection periods during which a value 

25 obtained from electric charge corresponding to the one or more phases of the 
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one set of phases does not exceed a value predetermined based on the 
saturation level, and is one detection period during which the value obtained 
from electric charge becomes maximum of that of the one or more detection 
periods. In this configuration, there is a possibility of which the distance 

5 value can be calculated even when the first specific detection period is not 
selected. Also, when the first specific detection period is shorter than the 
prescribed length, it is possible to utilize a value of available electric charge 
obtained during the detection period longer than the first specific detection 
period. As a result, it is possible to suppress influence of the shot noise to 

10 improve the measurement accuracy. 

In another enhanced embodiment, the image construction stage 
calculates a comparison value by applying at least a value obtained from one 
set of electric charges picked up by the sensor control stage to a function with 
respect to the quantity of light received by each of the photosensitive units. 

15 The image construction stage then compares the comparison value with a 
prescribed threshold value to select the specific detection period. In this 
configuration, for example, if the comparison value is smaller than the 
prescribed threshold value, a long detection period can be selected as the 
specific detection period out of the different detection periods. Also if the 

20 comparison value is larger than the prescribed threshold value, a short 
detection period can be selected as the specific detection period out of the 
different detection periods. 

In an alternate embodiment, the comparison value is a mean value 
with respect to each electric charge corresponding to each quantity of light 

25 received by each of the photosensitive units during one or plural periods of 
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the specific frequency. In this configuration, it is possible to consider 
intensity modulated light from the object space and ambient light component 
to distinguish whether or not the light detecting element is saturated. 

In another enhanced embodiment, when a distance value for a 
5 specific image element in the range image is not calculated, the image 
construction stage allocates an alternate value to the specific image element. 
In this configuration, when the specific detection period cannot be selected 
out of the different detection periods with respect to the specific image 
element, it is possible to construct a range image without an error value by 
10 allocating the alternate value to the specific image element. 

In an alternate embodiment, the alternate value is a past distance 
value for the specific image element. In this configuration, if the range 
image sensor is located in environment in which change of distance is 
comparatively few, a proper distance value can be allocated to the specific 
15 image element. 

In another alternate embodiment, the alternate value is a mean 
value with respect to a distance value of each image element around the 
specific image element. In this configuration, a proper distance value with 
continuity can be allocated to the specific image element. 
20 In other enhanced embodiment, the image construction stage 

calculates one set of integrating electric charges at every image element in 
the range image and calculates a distance value for each image element in 
the range image based on each one set of integrating electric charges. The 
one set of integrating electric charges is obtained that each electric charge in 
25 a plurality of the specific detection periods is added up at every same phase 
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of said one set of phases. In this configuration, since it is possible to 
comparatively reduce the quantity of light received in each specific detection 
period, saturation of the light detecting element can be prevented. 

In other enhanced embodiment, the light detecting element includes 

5 pixels each of which is consisted of two or more neighboring photosensitive 
units in the photosensitive units. Each of the pixels mainly generates and 
integrates two or more electric charges at its two or more neighboring 
photosensitive units in synchronization with two or more phases of each 
group obtained by dividing the one set of phases into groups each of which 

10 includes two or more phases, respectively. The sensor control stage controls 
to change synchronization timing of each integration period of two or more 
neighboring photosensitive units in each of the pixels with respect to each 
phase of a corresponding group of the groups so as to mutually interchange 
each phase of the corresponding group with respect to the neighboring 

15 photosensitive units. The sensor control stage also controls to pick up each 
electric charge mainly generated and integrated during each integration 
period corresponding to each phase of the each group at every pixel after at 
least the specific detection period of the different detection periods. The 
image construction stage combines each electric charge picked up at every 

20 pixel by the sensor control stage with one set of electric charges 
corresponding to the one set of phases. The image construction stage then 
calculates a distance value for each image element in the range image based 
on the one set of electric charges. In this configuration, it is possible to 
secure reliability of distance value obtained from each pixel consisted of the 

25 neighboring photosensitive units, since it is possible to almost remove error 
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caused by difference of each position of the neighboring photosensitive units. 

In other enhanced embodiment, a plurality of neighboring 
photosensitive units are set as a unit of arithmetic. The sensor control 
stage changes an integration period of each photosensitive unit of the unit of 

5 arithmetic to an integration period of a different phase of the one set of 
phases at every specific detection period. The image construction stage 
makes a value of an image element in the range image out of distance found 
by using an electric charge added up a period of time in which each 
photosensitive unit receives light from the object space. The light is 

10 received same number of times each in all integration periods of the phase. 
In this configuration, when a different photosensitive unit receives light 
during an integration period in synchronization with a different phase of the 
modulation signal in a specific detection period, position information of the 
plurality of photosensitive units of the unit of arithmetic is not included in 

15 an electric charge added up in order to find the distance. Therefore, 
reliability of the obtained distance becomes high. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Preferred embodiments of the invention will now be described in 
20 further details. Other features and advantages of the present invention 
will become better understood with regard to the following detailed 
description and accompanying drawings where: 

FIG. 1 is a block diagram of a range image sensor of a first 
embodiment according to the present invention; 
25 FIG. 2 shows operation of the range image sensor of the first 
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embodiment; 

FIG. 3 A shows a region corresponding to one photosensitive unit in 
the range image sensor of the first embodiment; 

FIG. 3B shows the region corresponding to the photosensitive unit; 
5 FIG. 4 is a schematic diagram of a pickup unit in the range image 

sensor of the first embodiment; 

FIG. 5A shows operation of a range image sensor of a second 
embodiment according to the present invention; 

FIG. 5B shows another operation of the range image sensor of the 
10 second embodiment; 

FIG. 6 shows operation of the range image sensor of the second 
embodiment; 

FIG. 7 shows operation of the range image sensor of the second 
embodiment; 

15 FIG. 8 A shows operation of an alternate embodiment; 

FIG. 8B shows another operation of the alternate embodiment; 

FIG. 9 is a block diagram of an image construction stage in a range 
image sensor of a fifth embodiment according to the present invention; and 

FIG. 10 shows operation of a range image sensor of a sixth 
20 embodiment according to the present invention. 

BEST MODE FOR CARRYING OUT THE INVENTION 

FIG. 1 shows a range image sensor of a first embodiment according to 
the present invention. The sensor comprises a light source 11, an optical 
25 system 12, a light detecting element 13, a sensor control stage 14 and an 
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image construction stage 15. 

The light source 11 is constructed with, for example, a LED array 
arranged on a plane, a semiconductor laser and a divergent lens, or the like, 
in order to provide enough intensity of light. The source 1 1 as shown in FIG. 

5 2 modulates intensity l { of light according to a modulation signal of a specific 
frequency from the sensor control stage 14 to emit sinusoidal 
intensity- modulated light toward an object space. However, not limited to 
this, intensity waveform of the intensity modulated light may be a shape 
such as a triangular wave, saw tooth wave or the like. Also, the source 1 1 

10 may comprise an infrared LED array, an infrared semiconductor laser and a 
divergent lens, or the like. 

The optical system 12 is a receiving optical system constructed with, 
for example a lens and so on, and condenses light from the object space into a 
receiving surface (each photosensitive unit 131) of the light detecting 

15 element 13. For example, the system 12 is disposed so as to orthogonalize 
its optical axis and the receiving surface of the element 13. However, the 
system 12 may comprise a lens, an infrared transmission filter and so on. 

The light detecting element 13 is formed in a semiconductor device 
and includes photosensitive units (each of which is denoted by 131), 

20 sensitivity control units (132), integration units (133) and a pickup unit 134. 
Each photosensitive unit 131, each sensitivity control unit 132 and each 
integration unit 133 constitute a two-dimensional photosensitive array 
disposed to face the object space through the optical system 12. 

Each photosensitive unit 131 as shown in FIGs. 3A and 3B is formed 

25 as a photosensitive element of, for example, a 100 x 100 photosensitive array 
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by an impurity doped semiconductor layer 13a in a semiconductor substrate. 
The unit 131 generates an electric charge of quantity in response to a 
quantity of light from the object space at the photosensitivitysensitivity 
controlled by a corresponding sensitivity control unit 132. For example, the 

5 semiconductor layer 13a is n-type and the generated electric charge is 
derived from electrons. 

When the optical axis of the optical system 12 is at right angles to the 
receiving surface, it is possible to regard the optical axis and both axes of 
vertical (length) direction and horizontal (breadth) direction of the receiving 

10 surface as three axes of an orthogonal coordinates system. Also, the center 
of the system 12 is regarded as the origin of the orthogonal coordinates 
system. In this case, each photosensitive unit 131 generates an electric 
charge of quantity in response to a quantity of light from direction indicated 
by angles of azimuth and elevation. When at least a physical object is in the 

15 object space, the light emitted from the light source 1 1 is reflected at the 
physical object and then received by each unit 131. Therefore, the unit 131 
as shown in FIG. 2 receives the intensity modulated light delayed by the 
phase \\f corresponding to the out and return distance up to the physical 
object and then generates an electric charge of quantity in response to its 

20 intensity 12. The intensity modulated light is represented by 

I 2 -sin(cdt-iif) + C, (1) 
where CD is an angular frequency and C is a value obtained by adding a mean 
value of intensity 12 to a value of ambient light component. 

The sensitivity control unit 132 is constructed with control electrodes 

25 (13b) layered on a surface of the semiconductor layer 13a through an 
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insulation film (oxide film) 13e. The unit 132 controls the sensitivity of a 
corresponding photosensitive unit 131 according to a sensitivity control 
signal from the sensor control stage 14. For example, as shown in FIGs. 3 A 
and 3B, the unit 132 is constructed with five control electrodes 13trl, 13b-2, 

5 13b-3, 13b-4 and 13b-5. Each electrode is applied with voltage (+V, OV) as 
the sensitivity control signal when the generated electric charge is derived 
from, for example, electrons. The +V is a predetermined positive voltage. 
The width size of the electrode 13b in right and left direction is set to 
approximately l|lm. The electrode 13b and the film 13e are formed of 

10 material with translucency with respect to the light of the light source 11. 

The integration unit 133 is comprised of a potential well (depletion 
layer) 13c which changes in response to the sensitivity control signal applied 
to corresponding each control electrode 13b. The unit 133 captures and 
integrates electrons (e) in proximity to the well 13c. Electrons not 

15 integrated in the unit 133 disappear by recombination with holes. 
Therefore, it is possible to control the photosensitivity -sensitivity of the light 
detecting element 13 by changing the size of region of the well 13c with the 
sensitivity control signal. For example, the sensitivity in a state of FIG. 3 A 
is higher than that in a state of FIG. 3B. 

20 The pickup unit 134, for example, as shown in FIG. 4 has similar 

construction to a CCD image sensor of frame transfer (FT) type. In an 
image pick up region Al constructed with the photosensitive units and a 
light-shielded storage region A2 next to the region Al, a semiconductor layer 
13a continuing as one in each vertical (length) direction is used as a transfer 

25 path of each electric charge along the vertical direction. The vertical 
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direction corresponds to the right and left direction ofFIGs. 3Aand 3B. 

The pickup unit 134 is constructed with the storage region A2, each 
transfer path, and a horizontal transfer register 13d which is a CCD and 
receives an electric charge from one end of each transfer path to transfer 

5 each electric charge along horizontal direction. Transfer of each electric 
charge from the image pick up region Al to the region A2 is executed at one 
time during a vertical blanking period. Namely, after each electric charge is 
integrated in each potential well 13c, a voltage pattern different from a 
voltage pattern of the sensitivity control signal is applied to each control 

10 electrode 13b as a vertical transfer signal. Accordingly, each electric charge 
integrated in each well 13c is transferred along the vertical direction. As to 
transfer from the horizontal transfer register 13d to the image construction 
stage 15, a horizontal transfer signal is supplied to the register 13d and each 
electric charge of one horizontal line is transferred during a horizontal period. 

15 However, the horizontal transfer register may transfer each electric charge 
along normal direction to the plane ofFIGs. 3A and 3B. 

The sensor control stage 14 is an operation timing control circuit and 
controls operation timing of the light source 11, each sensitivity control unit 
132 and the pickup unit 134. Namely, the stage 14 provides the source 11 

20 with the modulation signal of the specific frequency (e.g., 20MHz) to control 
change timing of the intensity of the intensity modulated light. Because 
transmission time of light for the abovementioned out and return distance is 
extremely short time, i.e., nanosecond level. 

The sensor control stage 14 also controls to synchronize an 

25 integration period of each photosensitive unit 1 3 1 with a specific phase (e.g., 
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each of one set of phases different from each other) of the modulation signal. 
In the first embodiment, the stage 14 as shown in FIGs. 2 and 3 A applies the 
voltage +V and the voltage OV to the control electrodes 13b-2 to 13b-4 and 
the electrodes 13b- land 13b- 5, respectively so as to synchronize start time of 

5 integration period Ti of each unit 131 with a phase angle of 0, 90, 180 or 270 
degrees of the modulation signal (c£, Ij). Accordingly, the sensitivity of each 
unit 131 is changed to a high sensitivity during the integration period 
shorter than one period of the specific frequency. Also, the stage 14 as 
shown in FIG. 3B applies the voltage +V and the voltage CN to the electrode 

10 13b-3 and the electrodes 13b- 1, 13b-2, 13b-4 and 13b-5, respectively in 
synchronization with a storing period which is a remaining period of said one 
period except the integration period. Accordingly, the sensitivity of each 
unit 131 is changed to a low sensitivity during the storing period. Therefore, 
in the element 13, part of electric charge generated by the unit 131 is 

15 integrated by the integration unit 133 (potential well 13c) on the high 
sensitivity, and then stored by the unit 133 on the low sensitivity. 

The sensor control stage 14 further controls to pick up each electric 
charge generated and stored in the light detecting element 13 from the 
element 13 in synchronization with a detection period corresponding to one 

20 or more periods of the specific frequency. Namely, the stage 14 supplies the 
vertical transfer signal to each control electrode 13b during the vertical 
blanking period and also supplies the horizontal transfer signal to the 
horizontal transfer register 13d during the horizontal period. For example, 
each electric charge generated and stored in the element 13 is picked up 

25 during a pickup period (see TR, to TR4 in FIG. 7) in synchronization with end 
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time of a detection period. 

The image construction stage 1 5 is constructed with, for example, a 
CPU, a storage device for storing a program and so on, etc. The stagel 8 
calculates a distance value for each image element in a range image based on 

5 each electric charge picked up by the sensor control stage 14. When at least 
a physical object is in the object space, the distance value represents distance 
up to the physical object. The stagel 5 then constructs the range image with 
a distance value for each image element. 

Principle of calculation of the distance value is explained. The 

10 phase (phase difference) \|/ of FIG. 2 corresponds to out and return distance 
between the receiving surface of the light detecting element 13 and the 
physical object in the object space. Therefore, by calculating the phase it 
is possible to calculate distance up to the physical object. The phase \\f and 
reflectance of the physical object seldom change in a period of time for 

15 integrating and picking up one set of electric charges. In this case, the 
phase \\f can be calculated from time integration values (e.g., Ti integration 
values Qo, Qi, Q2 and Q3) of a curve indicated by Eq. (1). Instantaneous 
values qo, qi, q2 and q 3 of the time integration values (quantities of light 
received) Qo, Qi, Q2 and Q3 are respectively given by 

20 qo = l2-sin(-iy) + C 

= - h'sinGy) + C, 
q { = 12-sin(7t/2 -\|/) + C 

= - 12'COsGl/) + C, 

q2 = l2'sin(n - \y) + C 
25 = 12- sin(\[/) + C, and 



WO 2006/030989 13 PCT/JP2005/0 17569 



q 3 = l2-sin(3n/2 - ijj) + C 
= • l2*cos(iir) + C. 

Therefore, the phase \\f is given by the following Eq. (2), and also in case of 
the time integration values it is possible to obtain the phase \\f by Eq. (2). 

5 \\f = tan -i {(q2 " qo) / (qi " q3)} (2) 

However, when the intensity modulated light is represented by 12- 
cos(0X - V|/) + C, the phase \|/ can be given by \\f = tan - ] {(qi - q 3 ) / (qo - q2)}. 
When a sign of the phase y is minus, order of each term of a denominator 
and a numerator in tan M...} may be interchanged each other or an absolute 

10 value may be used. 

In case the photosensitive unit 131 generates an electric charge in 
proportion to the quantity of light received, when the integration unit 133 
integrates an electric charge corresponding to Qo, the electric charge 
proportional to OOQo + B(Qi + Q2 + Q3) + BQx is integrated. The a is a 

15 sensitivity in each integration period Ti corresponding toQotoQ3, the (3 is a 
sensitivity in the storing period, and the Qx is a quantity of light received in 
the storing period. When the unit 131 also integrates an electric charge 
corresponding to Q2, the electric charge proportional to C1Q2 + B(Qo + Qi + Q2) 
+ BQx is integrated. Owing to Q2 - Qo = (a- (3)(Q2 - Qo) and Q 1 - Q3 = (a - 

20 B)(Qj * Q3), (Q2 - Qo) / (Qi - Q3) does not change in theory regardless of 
whether or not an unwanted electric charge is mixed in the integration unit 
(cf. Eq. (2)). Therefore, even if an unwanted electric charge is mixed in the 
integration unit, a phase \|/ for a corresponding image element does not 
change. 

25 In the first embodiment, the detection period includes different 
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detection periods. The different detection periods are, for example, a long 
detection period and a short detection period shorter than the long detection 
period. The short detection period is set to a period of time corresponding to 
one or more periods of the specific frequency. Also, each pixel in the image 

5 pick up region Al consists of, for example, four neighboring photosensitive 
units. And, the sensor control stage 14 controls to pick up one set of electric 
charges corresponding to the one set of phases at every pixel of the light 
detecting element (image pick up region Al) in synchronization with each of 
the different detection periods. However, not limited to this, each pixel may 

10 consist of one photosensitive unit and the sensor control stage may control to 
respectively pickup one set of electric charges corresponding to the one set of 
phases at every pixel in synchronization with at least four detection periods. 

The image construction stage 15 selects a specific detection period 
out of the different detection periods. The stage 15 then calculates a 

15 distance value for each image element in the range image based on one set of 
electric charges picked up at every pixel in synchronization with the specific 
detection period. The specific detection period is one of one or more 
detection periods during which the light detecting element 1 3 does not reach 
saturation and is one detection period during which a value related to the 

20 quantity of light received from the object space becomes maximum of that of 
the one or more detection periods. The value related to the quantity of light 
received is, for example, a value obtained from one set of electric charges. 

On account of this, the image construction stage 15 is constructed 
with hold units 151 andl52, a selection unit 153 and an arithmetic unit 154. 

25 The units 151 andl52 are constructed with a memory for storing each digital 
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value obtained by A/D converting each electric charge (analog signal). The 
unit 151 temporally holds one set of electric charges (digital values) picked 
up at every pixel in synchronization with the long detection period by the 
sensor control stage 14. The unit 152 temporally holds one set of electric 
charges (digital values) picked up at every pixel in synchronization with the 
short detection period by the stage 14. 

The selection unit 153 selects the specific detection period out of the 
long detection period and the short detection period at every one set of 
electric charges (digital values) in each of the hold units 151 and 152. If a 
value of one set of electric charges obtained from the unit 151 does not exceed 
a prescribed value corresponding to saturation level of the light detecting 
element 13, the long detection period corresponding to the unit 151 is 
selected as the specific detection period. If the value of one set of electric 
charges exceeds the prescribed value, the long detection period is not 
selected. In this case, if a value of one corresponding set of electric charges 
obtained from the unit 1 52 does not exceed the prescribed value, the short 
detection period corresponding to the unit 152 is selected as the specific 
detection period. 

Whenever the specific detection period is selected, the arithmetic 
unit 154, based on one set of electric charges (digital values) corresponding to 
the specific detection period, calculates a distance value for an image 
element corresponding to the pixel from which the one set of electric charges 
has been picked up. 

The operation of the first embodiment is now explained. The sensor 
control stage 14 controls the sensitivity of the light detecting element 13 to 



WO 2006/030989 



21 



PCT/JP2005/017569 



integrate an electric charge generated at each photosensitive unit 131 during 
each of the different detection periods into its integration unit 133. For 
example, when an electric charge corresponding to the time integration 
value (quantity of light received) Qois integrated into an unit 133, the stage 
14 raises the sensitivity of the unit 133 during the integration period Ti 
corresponding to Qo. The stage 14 also lowers the sensitivity of the unit 133 
during all (storing period) but the period Ti. Accordingly, it is possible to 
integrate and store the electric charge corresponding to Qo into the unit 133. 
Similarly, it is possible to integrate and store an electric charge 
corresponding to each time integration value into each unit 133. Also, a 
ratio of signal electric charge (electric charge corresponding to the intensity 
modulated light) to noise electric charge (electric charge corresponding to 
ambient light component and shot noise generated inside the element 13) 
can be made larger. Therefore, a large S/N ratio is obtained. 

In order to pick up an electric charge integrated into each integration 
unit 133, the sensor control stage 14 supplies the vertical transfer signal to 
each control electrode 13b during the vertical blanking period after the long 
detection period or the short detection period. The stage 14 also supplies 
the horizontal transfer signal to the horizontal transfer register 13d during 
the horizontal period. Accordingly, each electric charge integrated and 
stored during the long detection period is held by the hold unit 151. Also, 
each electric charge integrated and stored during the short detection period 
is held by the hold unit 152. 

The selection unit 153 selects the specific detection period out of the 
long detection period and the short detection period at every one set of the 
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electric charges in each of the units 151 and 152. Whenever the specific 
detection period is selected, the arithmetic unit 154, based on one set of 
electric charges for each pixel corresponding to the specific detection period, 
calculates a distance value for each pixel. Then, a range image is 
5 constructed. 

FIGs. 5A and 5B show operation of a range image sensor of a second 
embodiment according to the present invention. In the range image sensor 
of the second embodiment, two neighboring photosensitive units 23 land 231 
along the vertical direction are used as one pixel. Also, overflow drain is 

10 provided at every pixel. 

If a photosensitive unit generates the electric charges corresponding 
to Qo - Q3, resolution with respect to line-of-sight direction becomes high. 
But the problem of time difference is occurred because at least the specific 
detection period is required for each of electric charges corresponding toQo- 

15 Q3. Inversely, if four photosensitive units respectively generate the electric 
charges corresponding toQ Q - Q3, it is possible to pick up the electric charges 
corresponding to Qo • Q3 in synchronization with at least the specific 
detection period and therefore the time difference becomes small. However, 
the resolution with respect to line-of-sight direction becomes low. 

20 In the second embodiment as shown in FIGs. 5A and 5B, two 

photosensitive units 231 and 231 are utilized for one pixel in order to solve 
the problem. In FIGs. 3 A and 3B of the first embodiment, while an electric 
charge is generated at the photosensitive unit 131, two control electrodes 
13b-l and 13b-5 at both sides has function that forms potential barrier for 

25 preventing part of the electric charge from flowing in each neighboring 
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photosensitive unit 131. In the second embodiment, since any 
photosensitive unit 231 forms the barrier between potential wells 23c and 
23c of the neighboring photosensitive units 231 and 231, three control 
electrodes are provided with respect to each photosensitive unit 231. 

5 Therefore, six control electrodes 23b-l, 23b-2, 23b-3, 23b-4, 23b-5 and 23b-6 
are provided with respect to each pixel. 

The sensor control stage of the second embodiment controls to pick 
up one set of electric charges corresponding to the one set of phase of the 
modulation signal at every image element in the range image in 

10 synchronization with each of the different detection periods. 
Synchronization with each of the different detection periods as shown in FIG. 
7 is executed in order of a long detection period TLI, a short detection period 
Ts 1? a long detection period TL2 and a short detection period Ts2. 

The operation of the second embodiment is now explained. As 

15 shown in FIGs. 5 to 7, in the long detection period T L1 , the voltage +V and 
the voltage OV are applied to the control electrodes denoted by 23b- 1, 23b-2, 
23b-3 and 23b-5 and the electrodes denoted by 23b -4 and 23b-6, respectively 
during the integration period Ti corresponding to Qo. Then, the voltage +V 
and the voltage OV are applied to the electrodes denoted by 23b- 2, 23b-4, 

20 23b-5 and 23b- 6 and the electrodes denoted by 23b- 1 and 23b-3, respectively 
during the integration period Ti corresponding to Q2. Also, the voltage +V 
and the voltage OV are applied to the electrodes denoted by 23b-2 and 23b-5 
and the electrodes denoted by 23b- 1, 23b-3, 23b-4 and 23b-5, respectively 
during all (storing period) but the periods. Accordingly, both electric 

25 charges corresponding to Qo and Q2 are generated and integrated at every 
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pixel in synchronization with the period TLL 

In a pickup (readout) period TR,, the electric charges per pixel 
corresponding toQoand Q2are transferred from the image pickup region Al 
to the storage region A2. Namely, if the electric charge corresponding toQo 

5 is integrated into the potential well 23c corresponding to the electrodes 23b- 1, 
23V2 and 23b-3 and then the electric charge corresponding to Q 2 is 
integrated into the potential well 23c corresponding to the electrodes 23b-4, 
23b-5 and 23b-6, the electric charges corresponding to Qo and Q2 are picked 
up to be held at the hold unit corresponding to the long detection period. 

10 After this operation, the current detection period is changed to the 

short detection period Ts l5 and then operation same as the long detection 
period T L1 is executed. Accordingly, both electric charges corresponding to 
Qo and Q2 are generated and integrated at every pixel in synchronization 
with the period Ts v In a pickup period TR2 (equal to TRj), the electric 

15 charges corresponding to Q 0 and Q2 are picked up at every pixel to be held at 
the hold unit corresponding to the short detection period. 

After this operation, the current detection period is changed to the 
long detection period TL2 (equal to TLI) and also timing for applying each 
voltage pattern of FIGs. 5A and 5B is shifted 90 degrees, and then operation 

20 same as the long detection period TL { is executed. Accordingly, both electric 
charges corresponding to Q x and Q3 are generated and integrated at every 
pixel in synchronization with the period TL2. In a pickup period TR3 (equal 
to TR,), the electric charges corresponding to Q { and Q3 are picked up at 
every pixel to be held at the hold unit corresponding to the long detection 

25 period. 
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After this operation, the current detection period is changed to the 
short detection period Ts2 (equal toTsi), and then operation same as the long 
detection period TL2 is executed. Accordingly, both electric charges 
corresponding to Qj and Q3 are generated and integrated at every pixel in 

5 synchronization with the period Ts2. In a pickup period TR4 (equal to TRj), 
the electric charges corresponding to Q x and Q3are picked up at every pixel 
to be held at the hold unit corresponding to the short detection period. 

In the second embodiment, it is possible to obtain one set of electric 
charges corresponding to the long detection period and one set of electric 

10 charges corresponding to the short detection period at every operation 
periods Tpi and TP2- In this case, if the operation periods Tp 1 and Tp2 are set 
to a period of time shorter than 1/60 second, distance images are obtained at 
30 frames per second. 

In an alternate embodiment, the selection unit of the image 

15 construction stage obtains a value of maximum electric charge from each 
electric charge (digital value) in the hold unit corresponding to the long 
detection period at every image element in the range image. If the value of 
maximum electric charge does not exceed a maximum threshold value 
predetermined based on the saturation level of the light detecting element, 

20 the selection unit selects the long detection period as the specific detection 
period. If the value of maximum electric charge exceeds the maximum 
threshold value, the selection unit obtains a value of maximum electric 
charge from each electric charge in the hold unit corresponding to the short 
detection period. If the value of the maximum electric charge corresponding 

25 to the short detection period does not exceed the maximum threshold value, 
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the selection unit selects the short detection period as the specific detection 
period. Namely, the specific detection period is one of one or more detection 
periods during which a value of maximum electric charge for each of one set 
of electric charges held by each hold unit does not exceed the maximum 

5 threshold value, and is one detection period during which the value of 
maximum electric charge becomes maximum of that of the one or more 
detection periods. The short detection period is set to a period of time 
during which each value of maximum electric charge does not exceed the 
maximum threshold value under the environment to use. 

10 In another alternate embodiment, the selection unit of the image 

construction stage calculates a mean value with respect to one set of electric 
charges held by the hold unit corresponding to the long detection period at 
every image element in the range image. If the mean value does not exceed 
an average reference value predetermined based on the saturation level of 

15 the light detecting element, the selection unit selects the long detection 
period as the specific detection period. If the mean value exceeds the 
average reference value, the selection unit calculates a mean value with 
respect to one set of electric charges held by the hold unit corresponding to 
the short detection period. If the mean value corresponding to the short 

20 detection period does not exceed the average reference value, the selection 
unit selects the short detection period as the specific detection period. 
Namely, the specific detection period is one of one or more detection periods 
during which a mean value with respect to one set of electric charges held by 
each hold unit does not exceed the average reference value, and is one 

25 detection period during which the mean value becomes maximum of that of 
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the one or more detection periods. Each mean value "corresponds to an 
average with respect to each electric charge in response to the intensity 
modulated light from the object space and each electric charge in response to 
the ambient light component. Therefore, it is possible to distinguish 
5 whether or not the light detecting element is saturated. Each detection 
period is preferably set to a period of time during which fluctuation of the 
ambient light component can be ignored. 

In a preferred embodiment, instead of the voltage pattern of FIG. 5 A, 
the sensor control stage as shown as FIG. 8 A applies the voltage +V to the 
10 control electrodes 23b-l, 2 3b -2 and 23b- 3, applies voltage between +V and CN 
to the electrode 23b-5 and also applies the voltage OV to the electrodes 23b-4 
and 23b-6. Also, instead of the voltage pattern of FIG. 5B, the sensor 
control stage as shown as FIG. 8B applies the voltage between +V and OV to 
the electrode 13b-2, applies the voltage +V to the control electrodes 23b-4, 
15 23b-5 and 23b-6 and also applies the voltage OV to the electrodes 23b- 1 and 
23b-3. Thus if the potential well for mainly integrating an electric charge is 
deeper than the potential well for mainly storing an electric charge, an 
electric charge generated at the region corresponding to each electrode of the 
voltage OV becomes easy to flow into the deeper potential well. As a result, 
20 it is possible to reduce noise component which flows into the potential well 
for storing an electric charge. 

In other alternate embodiment, synchronization with each detection 
period is continuously executed a plurality of times. In this case, two 
electric charges picked up in synchronization with each detection period are 
25 added up or averaged at every same continuous detection period. For 
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example, the long detection period TLI and the pickup period TRj are 
continuously executed a plurality of times. The electric charges 
corresponding to Qo and Q2 picked up in synchronization with the long 
detection period TLI are added up or averaged at every continuous detection 

5 period TLI, respectively. According to this embodiment, since it is possible 
to comparatively reduce the quantity of light received in each detection 
period, saturation of the light detecting element can be prevented. 

In a third embodiment according to the present invention, when the 
specific detection period is not selected or a first specific detection period as 

10 the specific detection period is the short detection period shorter than 
prescribed length, the image construction stage selects a second specific 
detection period out of the different detection periods at every two phases of 
the one set of phases with respect to a specific image element for which the 
specific detection period is not selected or the short detection period is 

15 selected. 

In the example of FIG. 7, when a value of each electric charge in the 
long detection period TL { (QO, Q2) is smaller than a value (e.g., saturation 
threshold value) predetermined based on the saturation level of the light 
detecting element and also a value of each electric charge in the long 

20 detection period TL2 (QI, Q3) is larger than the saturation threshold value, 
the specific detection period is not selected or the short detection is selected. 
In this case, if a value of each electric charge in the short detection period Ts2 
(Qi, Q3) is smaller than the saturation threshold value, the image 
construction stage selects the long detection period TLI and the short 

25 detection period Ts2. Namely, the second specific detection period is one of 
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one or more detection periods during which a value obtained from electric 
charges corresponding to the two phases of the one set of phases does not 
exceed, for example, the saturation threshold value, and is one detection 
period during which the value obtained from electric charges becomes 
5 maximum of that of the one or more detection periods. 

The image construction stage then corrects two values obtained from 
one set of electric charges picked up with respect to the specific image 
element in synchronization with the second specific detection period based 
on a rate of length of the corresponding second specific detection period, and 
10 calculates one set of values. In case of the abovementioned example, a 
value of each electric charge in the short detection period Ts2 (Qi, Q3) is 
corrected based on the rate (T L I/TS2) of length of the corresponding short 
specific detection period Ts2. The relationship between the rate of length 
and a rate of each electric charge is previously obtained through 
15 experimentation. 

The image construction stage then calculates a distance value for the 
specific image element based on the one set of values. Accordingly, there is 
a possibility of which the distance value can be calculated even when the 
first specific detection period is not selected. Also, when the short detection 
20 period is selected as the first specific detection period, it is possible to utilize 
values of available electric charges obtained during the long detection period 
in stead of the short detection period. As a result, it is possible to suppress 
influence of the shot noise to improve the measurement accuracy. 

In a fourth embodiment according to the present invention, the 
25 image construction stage calculates a comparison value by substituting each 
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value corresponding to one set of electric charges picked up by the sensor 
control stage for each variable of function with respect to the quantity of 
light received by each photosensitive unit. The image construction stage 
then selects the specific detection period by comparing the comparison value 

5 with a prescribed threshold value. 

For example, the comparison value is a mean value with respect to 
each electric charge corresponding to each quantity of light received by each 
photosensitive unit during one or more periods of the specific frequency of 
the modulation signal, and is calculated by (Qo + Qi + Q2 + Q3) / 4. The 

10 mean value can be also calculated by (Qo + Q2 ) / 2 or (Qi + Q3) / 2 . When the 
ambient light component as shown in FIGs. 2 and 6 does not change during 
one or more periods of the specific frequency, the mean value is equivalent to 
the value C obtained by adding a mean value of intensity 12 to the ambient 
light component, and is a constant value. Therefore, the specific detection 

15 period can be selected based on the mean value. For example, if the mean 
value is larger than the prescribed threshold value, the short detection 
period is selected as the specific detection period. Otherwise, the long 
detection period is selected as the specific detection period. Also, if the 
mean value is smaller than the lower limit value smaller than the prescribed 

20 threshold value, the specific detection period is not selected. Because light 
from the object space is few and a proper distance value is not obtained. 

In a preferred embodiment, the sensor control stage controls to pick 
up one set of electric charges corresponding to the one set of phases of the 
modulation signal at every image element in the range image in 

25 synchronization with the specific detection period selected by the mean 
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value. 

In an alternate embodiment, when there is no ambient light 
component or ambient light component is known, amplitude A of intensity 
modulated light with intensity 12 is used as the comparison value. The 
5 amplitude A can be calculated from each quantity of light of Qo, Qi, Q2and 
Q3 and the following equation. 

A= (1 / 2)- {(Qo - Q2) 2 + (Qi - Q^ 2 ) 5 ' 2 

This equation is obtained from (Q 0 - Q2) 2 + (Qi - Q3) 2 = 4A 2 {sin 2 (\|/) + 
cos 2 (\|/)} = 4A 2 . If the amplitude A is larger than a prescribed threshold 

10 value, the short detection period is selected as the specific detection period. 
Otherwise, the long detection period is selected as the specific detection 
period. Also, if the amplitude A is smaller than a lower limit value smaller 
than the prescribed threshold value, the specific detection period is not 
selected since light from the object space is few and a proper distance value 

15 is not obtained. When ambient light component is not known, it is possible 
to distinguish whether or not the light detecting element is saturated by 
further utilizing the value C. 

In another alternate embodiment, a value obtained from the 
amplitude A and the value C, for example, a value (A/C) obtained by dividing 

20 the value C into the amplitude A is used as the comparison value. The 
value (A/C) corresponds to a ratio of received intensity modulated light to the 
mean value of the fourth embodiment. If the ratio is high, accuracy of 
distance value becomes high. Otherwise, accuracy of distance value 
becomes low. Therefore, if the value (A/C) is larger than a prescribed 

25 threshold value, the short detection period is selected as the specific 
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detection period. Otherwise, the long detection period is selected as the 
specific detection period. Also, if the value (A/C) is smaller than a lower 
limit value smaller than the prescribed threshold value, the specific 
detection period is not selected since a distance value of proper accuracy is 

5 not obtained. It is possible to distinguish whether or not the light detecting 
element is saturated by further utilizing the value C. 

FIG. 9 shows an image construction stage 45 in a range image sensor 
of a fifth embodiment according to the present invention. The image 
construction stage 45 comprises hold units 451 and 452, a selection unit 453 

10 and a arithmetic unit 454 as well as those of the first embodiment, and 
further comprises an exception processing unit 455. When a distance value 
for a specific image element in the range image is not calculated, the unit 
455 allocates an alternate value to the specific image element. 

For example, when a sum or maximum value of one set of digital 

15 values held by the hold unit corresponding to the short detection period is 
larger than a value predetermined based on the saturation level of the light 
detecting element or a sum or minimum value of one set of digital values 
held by the hold unit corresponding to the long detection period is smaller 
than a lower limit value necessary for calculation of distance value, the 

20 specific detection period is not selected for a corresponding image element. 
The alternate value is a prescribed value such as an average distance value 
up to the object space or the like. Accordingly, it is possible to construct a 
range image without lack of distance value. 

In an alternate embodiment, the alternate value is a past distance 

25 value for the specific image element. A term of validity is related to the past 
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distance value, and the past distance value for the specific image element is 
used as the alternate value as long as the term does not pass. 

In another alternate embodiment, the alternate value is a mean 
value with respect to a distance value of each image element around the 

5 specific image element. Even if the distance value for the specific image 
element is not calculated, there is a case that a distance value for each image 
element around the specific image element is calculated. In such case, by 
using the mean value as the alternate value, a proper distance value with 
continuity can be allocated to the specific image element. Also, the 

10 prescribed value, the past distance value or the mean value may be applied 
as the alternate value according to the order of priority. 

FIG. 10 shows operation of a range image sensor of a sixth 
embodiment according to the present invention. The range image sensor of 
the sixth embodiment is characterized by a sensor control stage and an 

15 image construction stage, and these stages operate to secure reliability of 
distance value obtained from each pixel consisted of two neighboring 
photosensitive units. 

Photosensitive units in a light detecting element of the sixth 
embodiment includes pixels each of which is consisted of two neighboring 

20 photosensitive units, as well as the second embodiment. Each of the pixels 
mainly generates and integrates two electric charges at its neighboring 
photosensitive units in synchronization with two phases of each group, 
respectively. The each group is obtained by dividing the one set of phases of 
the modification signal into two groups each of which includes two phases. 

25 Also, six control electrodes 63b- 1, 63b-2, 63b-3, 63b-4, 63b-5 and 63b-6 are 
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provided with respect to each pixel. 

Thus, when two photosensitive units are used as one pixel, there is a 
possibility that an error is included in a distance value obtained from the 
pixel due to difference of each position of the photosensitive units. For 
5 example, when a physical object to be detected in the object space has a step 
at the part corresponding to the position between the two photosensitive 
units, the error becomes large and accuracy of the distance value becomes 
low. 

Therefore, the sensor control stage of the sixth embodiment controls 

10 to change synchronization timing of each integration period of two 
neighboring photosensitive units in each pixel with respect to each phase of a 
corresponding group so as to mutually interchange each phase of the 
corresponding group with respect to the neighboring photosensitive units. 
The sensor control stage also controls to pick up each electric charge mainly 

15 generated and integrated during each integration period corresponding to 
each phase of each group at every pixel in synchronization with each of the 
different detection periods. 

In the long detection periods TLII (QO, Q2) and T L1 2(Q2, Qo) of FIG. 10, 
the sensor control stage changes synchronization timing of the integration 

20 period of photosensitive unit corresponding to the control electrodes 63b-l to 
63b-3 in each pixel with respect to the phase corresponding to Qo of TLII so as 
to interchange with the phase corresponding to Q2 with respect to the 
photosensitive unit. The sensor control stage also changes synchronization 
timing of the integration periods of photosensitive unit corresponding to the 

25 electrodes 63b~4 to 63b-6 in each pixel with respect to the phase 
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corresponding to Q2 of TUTT so as to interchange with the phase corresponding 
to Qo with respect to the photosensitive unit. In short, synchronization 
timing of each integration period of two neighboring photosensitive units in 
each pixel with respect to each phase of a corresponding group is changed so 
5 as to mutually interchange each phase of the corresponding group with 
respect to the neighboring photosensitive units. Besides, the sensor control 
stage picks up each electric charge mainly generated and integrated during 
each integration period corresponding to Qo and Q2 at every pixel in 
synchronization with the long detection periods TLII. Each electric charge is 
10 picked up during the pickup period TR n . The sensor control stage also picks 
up each electric charge mainly generated and integrated during each 
integration period corresponding to Q2 and Qo at every pixel in 
synchronization with the long detection periods TLI2. Each electric charge 
is picked up during the pickup period TRI2. 
15 In the long detection periods TL21 (QI, Q3) and TL22 (Q3, Qi) of FIG. 10, 

the sensor control stage changes synchronization timing of the integration 
period of photosensitive unit corresponding to the control electrodes 63b -1 to 
63b*3ineach pixel with respect to the phase corresponding to Qi of TL2I so as 
to interchange with the phase corresponding to Q3 with respect to the 
20 photosensitive unit. The sensor control stage also changes synchronization 
timing of the integration period of photosensitive unit corresponding to the 
electrodes 63b-4 to 63b-6 in each pixel with respect to the phase 
corresponding to Q 3 of TL21 so as to interchange with the phase 
corresponding to Qi with respect to the photosensitive unit. Besides, the 
25 sensor control stage picks up each electric charge mainly generated and 
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integrated during each integration period corresponding to Qj and Q3 at 
every pixel in synchronization with the long detection periods TL2L Each 
electric charge is picked up during the pickup period Tim. The sensor 
control stage also picks up each electric charge mainly generated and 

5 integrated during each integration period corresponding to Q3 and Q x at 
every pixel in synchronization with the long detection periods TL22. Each 
electric charge is picked up during the pickup period TR32. 

In the short detection periods corresponding to the long detection 
periods TLI1, TLI2, TL2I and TL22, the sensor control stage executes the same 

10 process as that in the long detection periods. 

The image construction stage of the sixth embodiment combines each 
electric charge picked up at every pixel by the sensor control stage with one 
set of electric charges corresponding to the one set of phases of the 
modulation signal. The image construction stage then calculates a distance 

15 value for each pixel based on the one set of electric charges. For example, a 
value of electric charge corresponding to Qo, Q l5 Q2 or Q3 is combined with 
the value of corresponding electric charge in one set of electric charges as a 
sum or mean value. In this case, the operation period Tpj corresponds to 
one frame of the range image. 

20 Thus, since each phase of each group is mutually interchanged with 

respect to each neighboring photosensitive units, it is possible to secure 
reliability of distance value obtained from each pixel consisted of two 
neighboring photosensitive units. 

In an alternate embodiment, the image construction stage calculates 

25 one set of integrating electric charges at every image element in the range 
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image and calculates a distance value for each image element in the range 
image based on each one set of integrating electric charges. The one set of 
integrating electric charges is obtained that each electric charge in a 
plurality of detection periods (e.g., specific detection periods) is added up at 
5 every same phase of the one set of phases. In this embodiment, since it is 
possible to comparatively reduce the quantity of light received in each 
specific detection period, saturation of the light detecting element can be 
prevented. 

In another alternate embodiment, a plurality of neighboring 

10 photosensitive units are set as a unit of arithmetic. The sensor control 
stage changes an integration period of each photosensitive unit of the unit of 
arithmetic to an integration period of a different phase of the one set of 
phases at every specific detection period. The image construction stage 
makes a value of an image element in the range image out of distance found 

15 by using an electric charge added up a period of time in which each 
photosensitive unit receives light from the object space. The light is 
received same number of times each in all integration periods of the phase. 
In this embodiment, when a different photosensitive unit receives light 
during an integration period in synchronization with a different phase of the 

20 modulation signal in a specific detection period, position information of the 
plurality of photosensitive units of the unit of arithmetic is not included in 
an electric charge added up in order to find the distance. Therefore, 
reliability of the obtained distance becomes high. 

Although the present invention has been described with reference to 

25 certain preferred embodiments, numerous modifications and variations can 
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be made by those skilled in the art without departing from the true spirit 
and scope of this invention. 

For example, in stead of the similar construction to the CCD image 
sensor of FT type, it is possible to use similar construction to that of interline 
5 transfer (IT) or frame interline transfer (FIT) type. 



